non-linear diversity

Non-linear diversity refers to techniques that attempt to create many different kinds of non-linear functions of basic features in order that higher level processing can use simpler forms of algorithms. Many real problems are non-linear, but many of the simplest and most explainable algorithms are linear. Creating sufficient non-linear diversity at early stages of a machine learning pipeline means that linear or at least less complex algorithms can be used at later stages.

Different methods can be used to generate non-linear diversity: support vector machines (SVM) use particular families of non-linear function (called a kernel) that separate out local and non-local features; reservoir computing uses a variety of physical or bilogcal proceses; and {[deep neural netowkrs}} have sufficjent layers that the early (typically wide) layers effectively create non-linear diversity for later (typically narrower) layers..